Our Commitments to Safe, Secure, and Trustworthy AI

Today, Scale will sign onto the Biden-Harris Administration’s voluntary commitments to ensure that AI is safe, secure, and trustworthy. These commitments are critical to the future of AI. The reality is that progress in frontier model capabilities must happen alongside progress in model evaluation and safety. This is not only the right thing to do, but practical. “To advance our knowledge and understanding of large language models (LLMs)–and therefore advance their capabilities–we must understand their weaknesses as much as their strengths. It is imperative for the tech industry to take proactive measures like these given the rapid pace of technological development.” –Alexandr Wang

The Scale team commends the White House’s leadership in bringing together companies that are shaping the future of responsible AI development. America’s continued technological leadership hinges on our ability to build and embrace the most cutting-edge AI across all sectors of our economy and government. This can only be achieved with collaboration between industry and government to create a broad understanding of the possibilities and limitations of the technology, so the benefits can be maximized and risks minimized. 

Safe, secure, and trustworthy AI has always been at the core of Scale’s mission. For more than seven years, we have built deep expertise in the entire lifecycle of developing, fine-tuning, and testing and evaluating AI. Our role as the data partner for the leading model developers has given us a unique vantage point across the AI ecosystem. It has allowed us to deeply understand the challenges that must be addressed, how various players across the industry are working to tackle them, and ultimately develop our own technology to test and evaluate LLMs. 

As a pioneer in the AI space, Scale is at the forefront of defining what it means for an LLM to be trusted and safe.

• Scale is committed to helping the broader AI community understand the benefits and safety risks and partnering with leading model builders to evaluate the safety of their models. Red teaming, in particular, is a core pillar of our approach to testing and evaluation. As announced by the White House’s Office of Science and Technology Policy, Scale was selected to develop the platform to evaluate leading generative AI systems for the largest red teaming exercise, to date, at DEFCON 31. 
• Scale published a vision for testing and evaluation and developed a platform to help establish a framework for an effective industry-wide approach, limit bias, and uphold the Responsible AI Principles, such as those outlined in the Biden Administration’s AI Bill of Rights. Successful adherence to and execution of these commitments is contingent on robust foundation model testing and evaluation. 
• Scale is grateful to join fellow industry leaders making these voluntary commitments to ensure the responsible development of AI technology. As we forge ahead, the company is committed to working with the Administration and public, private, and civil society to provide recommendations for concrete steps that can be taken to build a robust framework to unlock safe, secure, and trustworthy AI globally.